Hello everyone and welcome back to Computer Vision lecture series.

This is lecture 9 part 3.

In this lecture we will continue talking about what we are discussing in the last lecture

about correspondence problem.

We saw that we could solve correspondence search using a similarity constraint.

So given left and right stereo image patterns we plot a scan line across both the images

and we start looking, we fix a point in the left image or a window in the left image and

start to look for that similar windows along the scan line in the right image.

And using a matching cost or a normal matching cost could be a sum of square distances or

a normalized correlation to find the matching window on the right scan line.

And once we, if we use the sum of square distances the value of this cost will be low.

Whereas if we use a normalized correlation the value will be high because the correlation

is high.

And we can choose this windows to be matches and therefore the pixel on the left in the

middle of the window matches to the pixel on the right in the middle of that same window.

Okay, but we also saw the correspondence problem here that if we plot an intensity profile

along the scan lines in the left and the right image even though we can see visually that

they are quite similar that we have made a match it is not easy to match them.

How we will take an example.

Let's say we found that these two scan lines are these two are the stereo image pairs and

these two are the scan lines left and right.

And we start or fix a feature point along the scan line maybe this one which looks quite

distinct and it has two corners here.

So it's a very distinct feature point on the left hand image and we try to find or match

this window in the right hand side of the image.

So we take this window on the left hand side and this is a feature point for us because

these are corners so they are very good and we try to find them along the scan line or

the api-polar line on the right image.

And the neighborhood of these kind of corresponding points will be similar in intensity patterns

and using the matching cost sorry using the similarity constraints we can find this proper

windows and therefore we can eventually map the points.

So what do we do?

We take an image band we draw another scan line we take a small tool to show these parallel

lines is the intention to show these parallel lines is to show that we are going to slide

our window along this two parallel lines in both the images.

And we are looking in the so the left image band is like this and we fix our window along

this distinct feature point.

And we try to find or map the intensity values when we do a matching in the right image band.

And we find that the intensity profile is quite distinct and it can be easily see that

the we can easily see that the cross correlation is somewhere here.

So on the right image band the window will be nicely correlated in this in this location

and this graph high point or the peak of this graph shows that.

So essentially we have calculated a proper correlation using this window matching window

matching approach.

However there are problems with this.

What if we choose a feature less window or our window size is not big enough or small

enough.

For example visually we can say here that because there is a neighborhood of this window

with which has this distinctive feature I can match it to the right hand side of the

image.